gitlab.com/jokerrs1/Sia@v1.3.2/doc/File Contract Negotiation.md (about) 1 File Contract Negotiation 2 ========================= 3 4 Securing data on Sia requires creating and revising file contracts in an 5 untrusted environment. Managing data on Sia happens through several protocols: 6 7 + Settings Request - the host sends the renter its settings. 8 9 + Revision Request - the renter will send the host a file contract id, and the 10 host will send the most recent file contract revision that it knows of for 11 that file contract, with the signatures. A challenge and response is also 12 performed to verify that the renter is able to create the signatures to 13 modify the file contract revision. 14 15 + File Contract Creation - no data is uploaded during the initial creation of a 16 file contract, but funds are allocated so that the file contract can be 17 iteratively revised in the future. 18 19 + File Contract Revision - an existing file contract is revised so that data 20 can be added to an arbitrary place, or removed from an arbitrary place. 21 22 + File Contract Renewal - an existing file contract is renewed, meaning that a 23 new file contract with a different id is created, but that has the same data. 24 New funds are added to this file contract, and it can now be modified 25 separately from the previous contract. 26 27 + Data Request - data is requested from the host by hash. 28 29 + (planned for later) Storage Proof Request - the renter requests that the host 30 perform an out-of-band storage proof. 31 32 + (planned for later) Metadata Request - the renter requests some metadata 33 about the file contract from the host, namely the list of hashes that compose 34 the file. This list of hashes is provided along with a cryptographic proof 35 that the hashes are valid. The proof is only needed if only a subset of 36 hashes are being sent. 37 38 A frequently seen construction is 'acceptance'. The renter or host may have the 39 opportunity to accept or reject a communication, which takes the form of a 40 string. The acceptance string is always the same, and any string that is not 41 the acceptance string is a rejection. The rejection string can include reasons 42 why the rejection occurred, but most not exceed 255 bytes. After a rejection, 43 the connection is always closed. 44 45 The protocols described below are numbered. The number indicates when the 46 communicator is switching. Each pair of numbers is a full round trip of 47 communications. 48 49 All communications attempt to support slow connections and Tor connections. Any 50 connection with a throughput below 100kbps may struggle to perform the uploads 51 and downloads, and any connection with a rountrip latency greater than 2 52 minutes may struggle to complete the protocols. 53 54 Settings Request 55 ---------------- 56 57 The host signs the settings request to prove that the connection has opened to 58 the right party. Hosts announce on the blockchain and perform burn, therefore 59 identity is important. 60 61 1. The renter makes an RPC to the host, opening a connection. The connection 62 deadline should be at least 120 seconds. 63 64 2. The host sends the renter the most recent copy of its external settings, 65 signed by the host public key. The connection is then closed. 66 67 Revision Request 68 ---------------- 69 70 The renter requests a recent revision from the host. Often, this request 71 precedes modifications. A file contract can only be open for revision with one 72 party at a time. To prevent DoS attacks, the party must authenticate here by 73 performing a challenge-response protocol during the revision request. Putting 74 this challenge-response requirement in the revision-request can help improve 75 privacy, though the host is under no cryptographic or incentive-based 76 obligation to preserve the privacy of the revision. 77 78 1. The renter makes an RPC to the host, opening a connection. The connection 79 deadline should be at least 120 seconds. The renter sends the file contract 80 id for the revision being requested. 81 82 2. The host writes 32 bytes of random data that the renter must sign for the 83 renter key in the corresponding file contract. 84 85 3. The renter returns the signed challenge. 86 87 4. The host verifies the signature from the renter and then sends the renter 88 the most recent file contract revision, along with the transaction 89 signatures from both the renter and the host. The connection is then closed. 90 91 File Contract Creation 92 ---------------------- 93 94 A few decisions were made regarding the file contract protocol. The first is 95 that the renter should not sign the file contract until the host has formally 96 accepted the file contract. The second is that the host should be the last one 97 to sign the file contract, as the renter is the party with the strong 98 reputation system. 99 100 Instead of sending a whole transaction each time, the transaction is sent 101 piecemeal, and only the new parts at each step are sent to the other party. 102 This minimizes the surface area of data for a malicious party to manipulate, 103 which means less verification code, which means less chances of having a bug in 104 the verification code. 105 106 The renter pays for the siafund fee on the host's collateral and contract fee. 107 If a renter opens a file contract and then never uses it, the host does not 108 lose money. This does put the renter at risk, as they may open up a file 109 contract and then watch the host leave, but the renter is spreading the risk 110 over communications with many hosts, and has a reputation system that will help 111 ensure that the renter is only dealing with upstanding hosts. 112 113 1. The renter makes an RPC to the host, opening a connection. The connection 114 deadline should be at least 360 seconds. 115 116 2. The host sends the renter the most recent copy of its settings, signed. If 117 the host is not accepting new file contracts, the connection is closed. 118 119 3. The renter sends a notice of acceptance or rejection. If the renter accepts, 120 the renter then sends a funded file contract transaction without a 121 signature, followed by the public key that will be used to create the 122 renter's portion of the UnlockConditions for the file contract. 123 124 4. The host sends an acceptance or rejection of the file contract. If the host 125 accepts, the host will add collateral to the file contract, and will send 126 the renter the inputs + outputs for the collateral, followed by any new 127 parent transactions. The length of any of these may be zero. 128 129 5. The renter indicates acceptance or rejection of the file contract. If the 130 renter accepts, the renter will sign the file contract and send the 131 transaction signatures to the host. The renter will also send a signature 132 for a no-op file contract revision that follows the file contract. 133 134 6. The host may only reject the file contract in the event that the renter has 135 sent invalid signatures, so the acceptance step is skipped. The host signs 136 the file contract and sends the transaction signatures to the renter, and 137 the host creates and sends a signature for the no-op revision that follows 138 the file contract. The connection is closed. 139 140 File Contract Revision 141 ---------------------- 142 143 1. The renter makes an RPC to the host, opening a connection. The minimum 144 deadline for the connection is 600 seconds. The renter then sends a file 145 contract ID, indicating the file contract that is getting revised during the 146 RPC. 147 148 2. The host will respond with a 32 byte challenge - a random 32 bytes that the 149 renter will need to sign. 150 151 3. The renter will sign the challenge with the renter key that protects the 152 file contract. This is to prove that the renter has access to the file 153 contract. 154 155 4. The host will verify the challenge signature, then send an acceptance or 156 rejection. If accetped, the host will send the most recent file contract 157 revision for the file contract along with the transaction signagtures that 158 validate the revision. The host will lock the file contract, meaning no 159 other changes can be made to the revision file contract until this 160 connection has closed. 161 162 A loop begins. The host sends the most recent revision of the host settings 163 to the renter, signed. The settings are sent after each iteration of the 164 loop to enable high resolution dynamic pricing for the host, especially for 165 bandwidth. 166 167 6. The renter may reject or accept the settings + revision. A specific 168 rejection message will gracefully terminate the loop here. The renter will 169 send an unsigned file contract revision followed by a batch of modification 170 actions which the revision pays for. Batching allows the renter to send a 171 lot of data in a single, one-way connection, improving throughput. The 172 renter will send a number indicating how many modifications will be made in 173 a batch, and then sends each modification in order. 174 175 A single modification can either be an insert, a modify, or a delete. An 176 insert is an index, indicating the index where the data is going to be 177 inserted. '0' indicates that the data is inserted at the very beginning, '1' 178 indicates that the data will be inserted between the first and second 179 existing sectors, etc. The index is followed by the 4MB of data. A modify is 180 an index indicating which sector is being modified, followed by an offset 181 indicating which data within the sector is being modified. Finally, some 182 data is provided indicating what the data in the sector should be replaced 183 with starting from that offset. The offset + len of the data should not 184 exceed the sector size of 4MB. A delete is an index indicating the index of 185 the sector that is being deleted. Each operation within a batch is atomic, 186 meaning if you are inserting 3 sectors at the front of the file contract, 187 the indexes of each should be '0', '1', '2'. 188 189 7. The host indicates either acceptance or rejection of the new revision. 190 191 8. The renter signs the revision and sends the signature to the host. 192 193 9. The host signs the revision and sends the signature to the renter. Both 194 parties submit the new revision to the transaction pool. The connection 195 deadline is reset to 600 seconds (unless the maximum deadline has been 196 reached), and the loop restarts. 197 198 File Contract Renewal 199 --------------------- 200 201 1. The renter makes an RPC to the host, opening a connection. The minimum 202 deadline for the connection is 600 seconds. The renter then sends a file 203 contract ID, indicating the file contract that is getting revised during the 204 RPC. 205 206 2. The host will respond with a 32 byte challenge - a random 32 bytes that the 207 renter will need to sign. 208 209 3. The renter will sign the challenge with the renter key that protects the 210 file contract. This is to prove that the renter has access to the file 211 contract. 212 213 4. The host will verify the challenge signature, then send an acceptance or 214 rejection. If accetped, the host will send the most recent file contract 215 revision for the file contract along with the transaction signagtures that 216 validate the revision. The host will lock the file contract, meaning no 217 other changes can be made to the revision file contract until this 218 connection has closed. The host sends the most recent revision of the host 219 settings to the renter, signed. If the host is not accepting new file 220 contracts, the connection is closed. 221 222 5. The renter either accepts or rejects the settings. If accepted, the renter 223 sends a funded, unsigned file contract to the host, containing the same 224 Merkle root as the previous file contract, and also containing a renewed 225 payout with conditional payments to the host to cover the host storing the 226 data for the extended duration. 227 228 6. The host will accept or reject the renewed file contract. If accepted, the 229 host will add collateral (and miner fees if desired) and send the inputs + 230 outputs for the collateral, along with any new parent transactions. The 231 length of any of these may be zero. 232 233 7. The renter will accept or reject the host's additions. If accepting, the 234 renter will send signatures for the transaction to the host. The renter will 235 also send a signature for a no-op file contract revision that follows the 236 file contract. 237 238 8. The host may only reject the file contract in the event that the renter has 239 sent invalid signatures, so the acceptance step is skipped. The host signs 240 the file contract and sends the transaction signatures to the renter, and 241 the host creates and sends a signature for the no-op revision that follows 242 the file contract. The connection is closed. 243 244 Data Request 245 ------------ 246 247 1. The renter makes an RPC to the host, opening a connection. The connection 248 deadline is at least 600 seconds. The renter will send a file contract id 249 corresponding to the file contract that will be used to pay for the 250 download. 251 252 2. The host will respond with a 32 byte challenge - a random 32 bytes that the 253 renter will need to sign. 254 255 3. The renter will sign the challenge with the public key that protects the 256 file contract being used to pay for the download. This proves that the 257 renter has access to the payment. 258 259 4. The host will verify the challenge signature, and then send an acceptance or 260 rejection. If accepted, the host will send the most recent file contract 261 revision followed by the signautres that validate the revision. The host 262 will lock the file contract, preventing other connections from making 263 changes to the underlying storage obligation. 264 265 A loop begins. The host sends the most recent external settings to the 266 renter, signed. The settings are sent each iteration to provide high 267 resolution dynamic bandwidth pricing. 268 269 5. The host will send the renter the most recent file contract revision, along 270 with the signatures that validate the revision. 271 272 A loop begins, which will allow the renter to download multiple batches of 273 data from the same connection. The host will send the host settings, and the 274 most recent file contract revision transaction. If there is no revision yet, 275 the host will send a blank transaction. The host is expected to always have 276 the most recent revision (the host signs last), the renter may not have the 277 most recent revision. 278 279 6. The renter will accept or reject the host's settings. If accepting, the 280 renter will send a file contract revision, unsigned, to pay for the download 281 request. The renter will then send the download request itself. 282 283 7. The host will either accept or reject the revision. 284 285 8. The renter will send a signature for the file contract revision. 286 287 9. The host sends a signature for the file contract revision, followed by the 288 data that was requested by the download request. The loop starts over, and 289 the connection deadline is reset to a minimum of 600 seconds.